Liquid-based cooling system for cooling a multi-component electronics system

ABSTRACT

A system for cooling an electronics system is provided. The cooling system includes a monolithic structure preconfigured for cooling multiple electronic components of the electronics system when coupled thereto. The monolithic structure includes multiple liquid-cooled cold plates configured and disposed in spaced relation to couple to respective electronic components; a plurality of coolant-carrying tubes metallurgically bonded in fluid communication with the multiple liquid-cooled cold plates, and a liquid-coolant header subassembly metallurgically bonded in fluid communication with multiple coolant-carrying tubes. The header subassembly includes a coolant supply header metallurgically bonded to coolant supply tubes and a coolant return header metallurgically bonded to coolant return tubes. When in use, the multiple liquid-cooled cold plates engage respective electronic components of the electronics system, and liquid coolant is distributed through the liquid-coolant header subassembly and plurality of coolant-carrying tubes to the cold plates for removal of heat generated by the respective electronic components.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.11/539,910, filed Oct. 10, 2006, entitled “Liquid-Based Cooling Systemfor Cooling a Multi-Component Electronics System,” by Campbell et al.,which is hereby incorporated herein by referenced in its entirety.

Further, this application contains subject matter which is related tothe subject matter of the following applications, each of which isassigned to the same assignee as this application and each of which ishereby incorporated herein by reference in its entirety:

-   “Hybrid Cooling System and Method for a Multi-Component Electronics    System”, Campbell et al., Ser. No. 11/539,902, filed Oct. 10, 2006    and published on Apr. 10, 2008 as U.S. Patent Publication No.    US-2008-0084667 A1;-   “Cooling System and Method for a Multi-Component Electronics System    Employing Conductive Heat Transport”, Campbell et al., Ser. No.    11/539,905, filed Oct. 10, 2006 and published on Apr. 10, 2008 as    U.S. Patent Publication No. US-2008-0084668 A1;-   “Method of Assembling a Cooling System for a Multi-Component    Electronics System”, Campbell et al, Ser. No. 11/539,907, filed Oct.    10, 2006 and published on Apr. 25, 2008 as U.S. Patent Publication    No. US-2008-0092387 A1;-   “Method and Apparatus for Mounting a Heat Sink in Thermal Contact    with an Electronic Component”, Colbert et al, Ser. No. 11/201,972,    filed Aug. 11, 2005 and published on Feb. 15, 2007 as U.S. Patent    Publication No. US-2007-0035937 A1; and-   “Heatsink Apparatus for Applying a Specified Compressive Force to an    Integrated Circuit Device”, Colbert et al, Ser. No. 11/460,334,    filed Jul. 27, 2006 and published on Jan. 31, 2008 as U.S. Patent    Publication No. US-2008-0024991 A1.

TECHNICAL FIELD

The present invention relates in general to cooling an electronicssystem, and more particularly, to a liquid-based cooling system forcooling a multi-component electronics system. Still more particularly,the present invention relates to a liquid-based cooling systemcomprising a monolithic structure preconfigured for cooling multipleheat generating electronic components of an electronics system, whereinthe monolithic structure includes multiple liquid-cooled cold platesdisposed is spaced relation and configured to couple to respective heatgenerating electronic components of the electronics system.

BACKGROUND OF THE INVENTION

The power dissipation of integrated circuit chips, and the modulescontaining the chips, continues to increase in order to achieveincreases in processor performance. This trend poses a cooling challengeat both the module and system level. Increased air flow rates are neededto effectively cool high power modules and to limit the temperature ofair exhausted into the computer center.

In many large server applications, processors along with theirassociated electronics (e.g., memory, disk drives, power, etc.), arepackaged in removable drawer configurations stacked or aligned within arack or frame. In other cases, the electronics may be in fixed locationswithin the rack or frame. Typically, the components are cooled by airmoving in parallel air flow paths, usually front-to-back, impelled byone or more air moving devices (e.g., fans or blowers). In some cases itmay be possible to handle increased power dissipation within a singledrawer by providing greater air flow, for example, through the use of amore powerful air moving device or by increasing the rotational speed(i.e., RPMs) of an existing air moving device. However, this approach isbecoming unmanageable at the frame level in the context of a computerinstallation (e.g., data center).

The sensible heat load carried by the air exiting the frame willeventually exceed the ability of room air conditioning to effectivelyhandle the load. This is especially true for large installations of“server farms” or large banks of computer frames close together. In suchinstallations, not only will the room air conditioning be challenged,but the situation may also result in recirculation problems with somefraction of the “hot” air exiting one frame being drawn into the airinlet of the same or a nearby frame. Furthermore, while the acousticnoise level of a powerful (or higher RPM) air moving device in a singledrawer may be within acceptable acoustic limits, because of the numberof air moving devices in the frame, the total acoustic noise at theframe level may not be acceptable. In addition, the conventionalopenings in the frame for the entry and exit of air flow make itdifficult, if not impossible to provide effective acoustic treatment toreduce the acoustic noise level outside the frame. Finally, as operatingfrequencies continue to increase, electromagnetic cross talk betweentightly spaced computer frames is becoming a problem largely due to thepresence of the openings in the covers.

Accordingly, there is a significant need for enhanced cooling mechanismsfor electronic components, individually and at all levels of packaging,including for example, rack-mounted or blade-mounted electroniccomponents of various large computer systems today.

SUMMARY OF THE INVENTION

The need to cool current and future high heat load, high heat fluxelectronic components requires development of aggressive thermalmanagement techniques, such as liquid-based cooling systems and methodsof fabrication. The concepts disclosed herein address the need forenhanced liquid-based cooling systems for facilitating cooling of amulti-component electronics system.

Briefly summarized, the present invention comprises in one aspect acooling system for cooling an electronics system. The cooling systemincludes a single piece, monolithic structure preconfigured for coolingmultiple heat generating electronic components of the electronics systemwhen coupled thereto. The single piece, monolithic structure includes:multiple liquid-cooled cold plates configured and disposed in spacedrelation to engage respective heat generating electronic components ofthe electronics system to be cooled; a plurality of coolant-carryingtubes metallurgically, rigidly, permanently, bonded in fluidcommunication with the multiple liquid-cooled cold plates; and aliquid-coolant header subassembly metallurgically, rigidly, permanently,bonded in fluid communication with multiple coolant-carrying tubes ofthe plurality of coolant-carrying tubes, the liquid-coolant headersubassembly including a coolant supply header metallurgically bonded influid communication with coolant supply tubes of the multiplecoolant-carrying tubes and a coolant return header metallurgicallybonded in fluid communication with coolant return tubes of the multiplecoolant-carrying tubes. When in use, the multiple liquid-cooled coldplates engage the respective heat generating electronic components ofthe multiple heat generating electronic components to be cooled, andliquid coolant is distributed by the liquid-coolant header subassemblythrough the plurality of coolant-carrying tubes and the multipleliquid-cooled cold plates for removal of heat generated by therespective heat generating electronic components.

In a further aspect, a cooled electronics system is provided. The cooledelectronics system includes an electronics drawer of an electronicsrack. The electronics drawer comprises a component layout containingmultiple heat generating electronic components to be cooled. The cooledelectronics system further includes a liquid-based cooling system forcooling the multiple heat generating electronic components of theelectronics drawer. The cooling system comprises a single piece,monolithic structure preconfigured for the electronics drawer andcoupled to the multiple heat generating electronic components of theelectronics drawer. The single piece, monolithic structure includes:multiple liquid-cooled cold plates preconfigured in spaced relation andcoupled to respective heat generating electronic components of theelectronics drawer; a plurality of coolant-carrying tubesmetallurgically, rigidly, permanently, bonded in fluid communicationwith the multiple liquid-cooled cold plates; and a liquid-coolant headersubassembly metallurgically, rigidly, permanently, bonded in fluidcommunication with the multiple coolant-carrying tubes of the pluralityof coolant-carrying tubes, the liquid-coolant header subassemblyincluding a coolant supply header metallurgically bonded in fluidcommunication with coolant supply tubes of the multiple coolant-carryingtubes and a coolant return header metallurgically bonded in fluidcommunication with the coolant return tubes of the multiplecoolant-carrying tubes. When operational, liquid coolant is distributedby the liquid-coolant header subassembly through the plurality ofcoolant-carrying tubes and the multiple liquid-cooled cold plates forremoval of heat generated by the respective heat generating electroniccomponents of the electronics drawer.

In a still further aspect, a cooled electronics system is provided whichincludes an electronics rack and a liquid-based cooling system. Theelectronics rack comprises at least one electronics drawer having acomponent layout containing multiple heat generating electroniccomponents to be cooled. The liquid-based cooling system is coupled tothe multiple heat generating electronic components of the electronicsdrawer, and is a single piece, monolithic structure preconfigured forcooling selected components of the electronics drawer. The single piece,monolithic structure includes: multiple liquid-cooled cold platespreconfigured in spaced relation and coupled to respective heatgenerating electronic components within the electronics drawer; aplurality of coolant-carrying tubes metallurgically, rigidly,permanently, bonded in fluid communication with the multipleliquid-cooled cold plates; and a liquid-coolant header subassemblymetallurgically, rigidly, permanently, bonded in fluid communicationwith multiple coolant-carrying tubes of the plurality ofcoolant-carrying tubes. The liquid-coolant header subassembly includes acoolant supply header metallurgically bonded in fluid communication withcoolant supply tubes of the multiple coolant-carrying tubes and acoolant return header metallurgically bonded in fluid communication withthe coolant return tubes of the multiple coolant-carrying tubes. Whenoperational, liquid coolant is distributed by the liquid-coolant headersubassembly through the plurality of coolant-carrying tubes and themultiple liquid-cooled cold plates for removal of heat generated by therespective heat generating electronic components of the electronicsdrawer.

Further, additional features and advantages are realized through thetechniques of the present invention. Other embodiments and aspects ofthe invention are described in detail herein and are considered a partof the claimed invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter which is regarded as the invention is particularlypointed out and distinctly claimed in the claims at the conclusion ofthe specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 depicts one embodiment of a conventional air-cooled electronicsframe with heat generating electronic components disposed in removableelectronics drawers;

FIG. 2 is a plan view of one embodiment of an electronics drawer layoutillustrating multiple electronic components to be cooled, in accordancewith an aspect of the present invention;

FIG. 3 is a partially exploded perspective view of an air-cooled heatsink apparatus, in accordance with an aspect of the present invention;

FIG. 4 is a partial perspective view of the air-cooled heat sinkapparatus of FIG. 3, in accordance with an aspect of the presentinvention;

FIG. 5 is a cross-sectional elevational view of an air-cooled heat sinkapparatus and electronic component assembly, taken (for example) alongline 5-5 of FIG. 3, in accordance with an aspect of the presentinvention;

FIG. 6 is a cross-sectional view of a portion of the air-cooled heatsink apparatus of FIGS. 3-5, illustrating a non-influencing fastenerarrangement in an actuated state, in accordance with an aspect of thepresent invention;

FIG. 6A is a cross-sectional view of the non-influencing fastener ofFIG. 6, shown in a non-actuated state, in accordance with an aspect ofthe present invention;

FIG. 7 is a flowchart of one embodiment of a method of mounting anair-cooled heat sink in thermal contact with one or more electroniccomponents, in accordance with an aspect of the present invention;

FIG. 8 is a plan view of the electronics drawer layout of FIG. 2illustrating one alternate embodiment of a cooling system for coolingcomponents of the electronics drawer, in accordance with an aspect ofthe present invention;

FIG. 9 depicts one detailed embodiment of a partially assembledelectronics drawer layout, wherein the electronics system includes eightheat generating electronic components to be actively cooled, each havinga respective liquid-cooled cold plate of a liquid-based cooling systemcoupled thereto, in accordance with an aspect of the present invention;

FIG. 10A depicts one embodiment of a liquid-cooled cold plate employedin the cooling system embodiment of FIG. 9, in accordance with an aspectof the present invention;

FIG. 10B depicts one embodiment of a liquid-coolant header subassemblyemployed in the cooling system embodiment of FIG. 9, in accordance withan aspect of the present invention;

FIG. 10C depicts multiple preconfigured coolant-carrying tubes employedin the cooling system embodiment of FIG. 9, in accordance with an aspectof the present invention;

FIG. 11 is a perspective view of one embodiment of a liquid-cooled coldplate and electronic component assembly, in accordance with an aspect ofthe present invention;

FIG. 12 is an exploded view of the liquid-cooled cold plate andelectronic component assembly of FIG. 11, in accordance with an aspectof the present invention;

FIG. 13 is a top plan view of one embodiment of a liquid-cooled coldplate (shown with the cover removed) for a cooling system, in accordancewith an aspect of the present invention; and

FIG. 13A is a cross-sectional elevational view of the liquid-cooled coldplate of FIG. 13, taken along line 13A-13A, in accordance with an aspectof the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

As used herein “electronics system” comprises any system containing oneor more heat generating components of a computer system or otherelectronics unit requiring cooling. The terms “electronics rack”,“electronics frame”, and “frame” are used interchangeably, and includeany housing, rack, compartment, blade chassis, etc., having heatgenerating components of a computer system or electronics system and maybe for example, a stand-alone computer processor having high, mid or lowend processing capability. In one embodiment, an electronics framecomprises multiple electronics drawers, each having multiple heatgenerating components disposed therein requiring cooling. “Electronicsdrawer” refers to any sub-housing, blade, book, drawer, node,compartment, etc., having multiple heat generating electronic componentsdisposed therein. Each electronics drawer of an electronics frame may bemovable or fixed relative to the electronics frame, with rack mountedelectronics drawers and blades of a blade center system being twoexamples of drawers of an electronics frame to be cooled.

“Electronic component” refers to any heat generating electroniccomponent of, for example, a computer system or other electronics unitrequiring cooling. By way of example, an electronic component maycomprise one or more integrated circuit dies and/or other electronicdevices to be cooled, including one or more processor dies, memory diesand memory support dies. As a further example, the electronic componentmay comprise one or more bare dies or one or more packaged dies disposedon a common carrier. As used herein, “primary heat generating component”refers to a primary heat generating electronic component within theelectronics system, while “secondary heat generating component” refersto an electronic component of the electronics system generating lessheat than the primary heat generating component to be cooled. “Primaryheat generating die” refers, for example, to a primary heat generatingdie or chip within a heat generating electronic component comprisingprimary and secondary heat generating dies (with a processor die beingone example). “Secondary heat generating die” refers to a die of amulti-die electronic component generating less heat than the primaryheat generating die thereof (with memory dies and memory support diesbeing examples of secondary dies to be cooled). As one example, a heatgenerating electronic component could comprise multiple primary heatgenerating bare dies and multiple secondary heat generating dies on acommon carrier. Further, unless otherwise specified herein, the term“liquid-cooled cold plate” refers to any conventional thermallyconductive structure having a plurality of channels or passagewaysformed therein for flowing of liquid coolant therethrough. In addition,“metallurgically bonded” refers generally herein to two components beingwelded, brazed or soldered together by any means.

As shown in FIG. 1, in rack-mounted configurations typical in the priorart, a plurality of air moving devices 111 (e.g., fans or blowers)provide forced air flow 115 needed to cool the electronic components 112within the electronics drawers 113 of the frame 100. Cool air is takenin through a louvered inlet cover 114 in the front of the frame andexhausted out a louvered outlet cover 116 in the back of the frame.

FIG. 2 illustrates one embodiment of a multi-component electronicsdrawer 213 having a component layout in accordance with an aspect of thepresent invention. Electronics drawer 213 includes one or more airmoving devices 211 (e.g., fans or blowers) which provide forced air flow215 across the multiple electronic components 212 within electronicsdrawer 213. Cool air is taken in through a front 231 of electronicsdrawer 213 and exhausted out a back 233 of the electronics drawer. Inthis embodiment, the multiple electronic components to be cooled 212include processor modules disposed below air-cooled heat sinks 220, aswell as (by way of example) multiple rows of memory support modules 232disposed between arrayed memory modules 230, such as air-cooled dualin-line memory module (DIMM) packages.

Electronic components are generally packaged using one or moreelectronic packages (i.e., modules) that include a module substrate towhich the device is electrically connected. In some cases, the moduleincludes a cap (i.e., a capped module) which seals the electronic devicewithin the module. In other cases, the module does not include a cap(i.e., is a bare die module).

Bare dies are generally preferred over capped modules from a thermalperformance perspective. In the case of a capped module, a heat sink istypically attached with a thermal interface between a bottom surface ofthe heat sink and a top surface of the cap, and another thermalinterface between a bottom surface of the cap and a top surface of theelectronic device. In the case of a bare die, a heat sink is typicallyattached with a thermal interface between a bottom surface of the heatsink and a top surface of the electronic device. Bare dies typicallyexhibit better thermal performance than capped modules because bare dieseliminate two sources of thermal resistance present in capped modules,i.e., the thermal resistance of the cap and the thermal resistance ofthe thermal interface between the cap and the electronic device.Accordingly, bare dies may be preferred for electronic components thathave high power dissipation.

Air-cooled heat sinks are attached to modules using a variety ofattachment mechanisms, such as clamps, screws and other hardware. Theattachment mechanism typically applies a force that maintains a thermalinterface gap, i.e., the thickness of the thermal interface extendingbetween the heat sink and the module. In the case of a capped module,the cap protects the electronic device from physical damage from theapplied force. In the case of a bare die, however, the applied force istransferred directly through the electronic device itself. Consequently,when bare dies are used, the attachment mechanism typically applies acompliant force to decrease stresses on the electronic component.

FIGS. 3-7 depict one enhanced mounting mechanism for holding anair-cooled heat sink in thermal contact with an electronic component.Generally stated, in this embodiment, the air-cooled heat sink apparatuscomprises a load frame having load springs and an open region thatexposes the electronic component. The load frame is mounted to a circuitboard on which the electronic component is mounted. The air-cooled heatsink is disposed on the load frame and has a main body in thermalcontact with the electronic component through a thermally conductivematerial. The air-cooled heat sink has load arms for engaging the loadsprings. A load plate extends between the load arms and has an actuationelement operative to displace the main body relative to the load plateand thereby resiliently deform the load springs and produce a load forcethat compresses the thermally conductive material to achieve a desiredthermal interface gap between the main body and the electroniccomponent. Non-influencing fasteners secure the air-cooled heat sink tothe load frame and maintain the desired thermal interface gap.

Referring to FIGS. 3-5, an air-cooled heat sink 220 is illustrated,which implements an improved process for mounting the heat sink onto theheat source, such as an electronic component. FIGS. 3-5 illustrate majorcomponents of an air-cooled heat sink apparatus 220 at a high level, andit should be understood that the number, type and configuration ofcomponents may vary depending upon the implementation. For example, theapparatus may contain a different number, type and configuration ofelectronic modules to be cooled.

As best shown in FIG. 3, air-cooled heat sink apparatus 220 includes twomain components, i.e., a load frame/spring assembly 302 and a heatsink/load arm assembly 304. Load frame/spring assembly 302 includes aload frame 306 and a pair of load springs 308. Load frame 306 ispreferably made of an alloy material chosen for its low creepproperties, such Zamak 8. Zamak 8, also known as ZA-8, is the trade namefor a zinc-based alloy, the primary components of which are zinc,aluminum, copper, and magnesium. Creep is the development over time ofadditional strains in a material. Creep depends on the magnitude of theapplied force and its duration, as well as the temperature and pressure.A material having high creep resistance is preferable in theconstruction of load frame 306 because creep deformation is to beavoided.

Load springs 308 are preferably made of an alloy material chosen for itshigh tensile strength properties, such as high strength music wire.Although two load springs 308 are shown in FIG. 3, those skilled in theart will appreciate that the present invention may be practiced with anynumber of load springs 308 (and load arms 310, which engage the loadsprings 308 as described below in the discussion of heat sink/load armassembly 304).

Load frame 306 is mounted on a printed circuit board 312. Referring toFIG. 5, fasteners such as screws 510 (two of which are denoted withdotted lines in FIG. 5) are used to attach load frame 306 to printedcircuit board 312. In one embodiment, four screws 510 (i.e., one neareach corner of load frame 306) pass through thru-holes in a backsidestiffener 512, an insulator 514 such as a polyimide, and printed circuitboard 312, and are received in threaded holes in load frame 306. Thisconfiguration advantageously allows access to screws 510 even when theheat sink/load arm assembly is attached to the load frame/springassembly.

Returning to FIG. 3, load frame 306 includes one or more open regions314 into which extends the heat source, e.g., an electronic component(not shown) mounted on printed circuit board 312. For example, a baredie may be mounted on printed circuit board 312 at the locationdesignated at the intersection of the cross-hairs shown in FIG. 3.

As shown in FIG. 3, load frame 306 includes four mounting projections316 to which the ends of load springs 308 are secured. Load frame 306also includes two downstop support projections 318 on which rest themid-sections of load springs 308.

One or more non-influencing fasteners 320 are used to secure heatsink/load arm assembly 304 to load frame/load arm assembly 302. By wayof example, four non-influencing fasteners 320 are mounted on load frame306. Each non-influencing fastener 320 is threaded into a boss 516 (FIG.5) of load frame 306. The non-influencing fasteners (NIFs) lock the heatsink in position without influencing the position of the heat sink.

Heat sink/load arm assembly 304 includes a heat sink 324 having a baseplate 326. Preferably, heat sink 324 is formed with fins, pins or othersimilar structures to increase the surface area of the heat sink andthereby enhance heat dissipation as air passes over the heat sink. It isalso possible for heat sink 324 to contain high performance structures,such as vapor chambers and/or heat pipes, to further enhance heattransfer. For example, heat sink 324 may contain one or more vaporchambers (not shown) charged with deionized water. Heat sink 324 may,for example, be formed of metal, such as copper or aluminum, or of otherthermally conductive material, such as graphite-based material.

As mentioned above, heat sink/load arm assembly 304 includes load arms310. Load arms 310 are hingedly attached to a U-channel load plate 328.Load arms 310 and U-channel load plate 328 may be made of stainlesssteel, for example, and be configured to provide minimal air flowimpedance across the fins of heat sink 324. For example, load arms 310have an open area through which air may flow. When heat sink/load armassembly 304 is attached to load frame/spring assembly 302, load arms310 engage load springs 308. This engagement is described in detailbelow with reference to FIGS. 4 & 5. In addition, when heat sink/loadarm assembly 304 is attached to load frame/spring assembly 302,non-influencing fasteners 320 are received in bore holes 330 in the heatsink's base plate 326. This non-influencing fastener arrangement isdescribed further below with reference to FIGS. 5-6A. To aid inalignment of heat sink/load arm assembly 304 with respect to loadframe/spring assembly 302, load frame 306 may include alignment pins332, which are received in corresponding alignment holes (not shown) inthe heat sink's base plate 326.

FIG. 4 is a perspective view of a heat transfer apparatus 220 withportions of heat sink 324 removed. FIG. 5 is a cross-sectional view ofheat transfer apparatus 220 engaging an electronic component assembly.As shown in FIGS. 4 & 5, an actuation mechanism applies a preload forceto heat sink 324 toward a semiconductor chip 502 (FIG. 5) to compress athermally conductive material 508 (FIG. 5) and achieve a desired thermalinterface gap between heat sink 324 and semiconductor chip 502. The maincomponents of the actuation mechanism include load frame 306, the loadframe's mounting projections 316, load springs 308, load arms 310, theload arms' hook portions 410, hinge pins 412, U-channel load plate 328,actuation screw 414, push plate 520, the push plate's guide pins 334,heat sink 324, and the heat sink's base plate 326. Referring to FIG. 3,load arms 310 each include a hook portion 410 that engages one of theload springs 308. Load arms 310 are hingedly attached to U-channel loadplate 328 by hinge pins 412. An actuation screw 414 is threaded throughU-channel load plate 328 to engage an underlying push plate 520 (FIG.5). Actuation screw 414 may be, for example, an M3 screw. Actuationscrew 414 is accessible for actuation from the top of U-channel loadplate 328. The distance between the U-channel plate and push plate 520is adjusted by turning actuation screw 414. This provides a controlledrate of loading. Those skilled in the art will recognize that otheractuation elements and techniques to provide a controlled rate ofloading are possible within the scope of the present invention, such ascamming, rocking and the like.

Still referring to FIG. 4, when the load frame/spring assembly and theheat sink/load arm assembly are brought together, hook portions 410 ofload arms 310 are engaged with load springs 308, and the actuationmechanism is actuated by turning actuation screw 414 in a direction toincrease the distance between U-channel load plate 328 and theunderlying push plate 520 (FIG. 5). Load springs 308 are deflected byactuation of the actuation mechanism. The geometric parameters of loadsprings 308, (i.e., the span, cross-section profile, and diameter) areoptimized for the allowable space within the application and therequired resulting load. Force is transmitted through the heat sink'sfins and base plate 326 onto the underlying semiconductor chip 502 (FIG.5). The force compresses a thermally conductive material 508 (FIG. 5)and achieves a desired thermal interface gap between heat sink's baseplate 326 and semiconductor chip 502.

Referring to FIG. 5, push plate 520 is affixed to heat sink 324. Forexample, push plate 520 may be soldered to heat sink 324 using, forexample, SAC 305 solder. Alternatively, push plate 520 may be affixed toheat sink 324 with a suitable adhesive, such as epoxy. Push plate 520may be made of stainless steel, for example. In one embodiment, pushplate 520 is affixed in a location directly above the heat source, withthe width of U-channel load plate 328 and push plate 520 substantiallycapturing the footprint of the heat source. This provides centroidalloading above the bare die, and thus provides substantially no edgestress on the die. As shown in FIG. 5, for example, push plate 520 isaffixed to multiple heat sink's fins lying above semiconductor chip 402.Although not shown in FIG. 5, additional modules residing on printedcircuit board 312 may be accommodated in open area 314 of load frame306. In such a case, push plate 520 may be affixed in a locationdirectly over the primary module, with the width of U-channel load plate328 and push plate 520 substantially capturing the footprint of theprimary module.

As shown in FIGS. 3 and 4, the push plate includes guide pins 334 thatextend through corresponding holes in U-channel load plate 328. Thepurpose of guide pins 334 is to align push plate 520 relative toU-channel load plate 328.

As shown in FIG. 5, in one embodiment, the heat generating electroniccomponent comprises one or more bare dies, including a semiconductorchip 502, a module substrate 504, and an electronic connector 506.However, those skilled in the art will appreciate that the presentinvention may be practiced using other types of heat sources such as oneor more capped modules and/or other electronic components. The bare dieshown in FIG. 5 is a single-chip module (SCM); however, those skilled inthe art will recognize that the spirit and scope of the presentinvention is not limited to SCMs. For example, those skilled in the artwill recognize that the present invention may be practiced using one ormore multi-chip modules (MCMs), or a combination of MCMs, SCMs and/orother electronic components/heat sources.

It is significant to note that the present invention allows a singleheat transfer apparatus to accommodate one or more modules havingdifferent footprints. Previous solutions required qualification ofindividual modules based on differences in footprint. The presentinvention overcomes this drawback.

The bare die is conventional. Semiconductor chip 502 is electricallyconnected to module substrate 504. Electronic connector 506, whichelectrically connects printed circuit board 312 to module substrate 504,may be a pin grid array (PGA), a ceramic column grid array (CCGA), aland grid array (LGA), or the like.

In some cases, electronic connector 506 may be susceptible to beingcrushed by the force applied by the actuation mechanism. This isproblematic not only from the perspective of possible damage toelectronic connector 506, but it also throws off the planarity of thestack (i.e., the module substrate 504 and semiconductor chip 502)relative to the heat sink's base plate which causes thermally conductivematerial 508 to form an uneven thermal interface gap. In such cases, oneor more crush protection elements 522 (denoted with a dotted line inFIG. 5) may be inserted along peripheral portions of module substrate504 between the bottom of module substrate 504 and the top of printedcircuit board 312. The crush protection elements 522 may be made of amaterial such as a polythermal plastic or the like.

Referring to FIG. 5, thermal interface 508 is made of a thermallyconductive material such as thermal gel, grease, paste, oil, or otherhigh thermal conductivity material. For example, thermal interface 508may be made of Shin-Etsu gel or grease with aluminum and/or zinc oxidespheres. Typically, thermal interface 508 is relatively thin so that itmay easily transfer heat away from semiconductor chip 502 towards theheat sink's base plate 326. The thickness of thermal interface 508extending between the bottom of the heat sink's base plate 326 and thetop surface of semiconductor chip 502 is referred to as the thermalinterface gap. As one example, the thermal interface gap is about 1.2mil.

Thermally conductive material 508 is dispensed on semiconductor chip 502prior to bringing the load frame/spring assembly and the heat sink/loadarm assembly together. To protect semiconductor 502 as these assembliesare initially brought together, a viscoelastic foam pad 530 may beinterposed between the lower surface of the heat sink's base plate 326and the upper surface of load frame 306.

Those skilled in the art will appreciate that the actuation mechanismshown in FIGS. 4 and 5 is exemplary, and that other actuation mechanismsmay be used to apply the preload force within the spirit and scope ofthe present invention. According to one embodiment of the presentinvention, once the preload force is applied to achieve the desiredthermal gap, irrespective of the actuation mechanism that applied thepreload force, one or more non-influencing fasteners are actuated tosecure the heat sink to the load frame and maintain the desired thermalgap.

As shown in FIG. 5, when the heat sink/load arm assembly is attached tothe load frame/spring assembly, non-influencing fasteners 320 arereceived in bore holes 330 in the heat sink's base plate 326. Once theactuation mechanism applies the preload force to achieve the desiredthermal interface gap, non-influencing fasteners 320 are actuated tosecure heat sink 324 to load frame 306 and maintain the desired thermalgap. One embodiment of a non-influencing fastener arrangement is shownin more detail in FIGS. 6 and 6A. FIG. 6 shows a non-influencingfastener 320 in an actuated state, while FIG. 6A shows non-influencingfastener 320 in a non-actuated state. Non-influencing fastener 320includes a screw 610 that is threaded into one of the bosses 516 of loadframe 306. Captivated on screw 610 are a split taper ring 620 and asolid taper ring 630. Preferably, the taper of split taper ring 620matches that of solid taper ring 630. Non-influencing fastener 320 isaccessible through bore hole 330 in the heat sink's base plate 326, andis actuated by turning screw 610 into the load frame's boss 516 so thatsplit taper ring 620 is expanded against the wall of bore hole 330 inthe heat sink's base plate 326. Non-influencing fasteners 320 areadvantageous because they can be actuated without significantly alteringthe thermal interface gap, as would be the case with a conventionalfastener.

FIG. 7 is a flow diagram of a method 700 for mounting a heat sink inthermal contact with an electronic component according to one embodimentof the present invention. Method 700 sets forth one order of steps. Itshould be understood, however, that the various steps may occur at anytime relative to one another. Initially, the bare die is soldered to theprinted circuit board 710. If a crush protection element is desired,then the crush protection element is inserted along peripheral portionsof the module substrate between the bottom of the module substrate andthe top of printed circuit board 720. The load frame is attached to theprinted circuit board 730. Thermally conductive material is dispensed onthe semiconductor chip 740. Next, the heat sink/load arm assembly isaligned and brought into contact with the load frame/spring assembly750. During step 750, the hook portion of each load arms is brought intoengagement with one of the load springs.

Method 700 continues with the application of a preload force using theactuation mechanism to set the thermal interface gap 760. During step760, the actuation screw is turned an appropriate amount to apply apreload force (e.g., 40 lbs) that provides the desired thermal interfacegap (e.g., 1.2 mil). In other words, some of the thermally conductivematerial is squeezed-out by the preload force to provide the desiredthermal gap. Once this point is reached, the assembly may optionally bethermally cured to set the thermal interface gap. Next, thenon-influencing fasteners are actuated to secure the heat sink to theload frame and maintain the desired thermal gap (step 770). Preferably,an appropriate torque is applied to the non-influencing fasteners usingan X-pattern sequence to minimize the application of any stresses.

Thermal sensors may be used to measure the thermal interface gapachieved by method 700. If the desired thermal interface gap is notachieved, then the unit may be simply reworked by removing the heatsink/load arm assembly from the load frame/spring assembly, and cleaningthe thermally conductive material from the semiconductor chip, andreturning to step 740.

As noted above, in order to provide greater performance, it willeventually be necessary to increase processor chip powers beyond thepoint where forced air-cooling is feasible as a solution. To meet thisincreased cooling demand, a liquid-based cooling system is providedherein, with a liquid-cooled cold plate physically coupled to eachprimary heat generating component to be cooled. FIG. 8 is a depiction ofthe electronics drawer component layout of FIG. 2, shown with such acooling system.

More particularly, FIG. 8 depicts one embodiment of an electronicsdrawer 813 component layout wherein one or more air moving devices 811provide forced air flow 815 to cool multiple components 812 withinelectronics drawer 813. Cool air is taken in through a front 831 andexhausted out a back 833 of the drawer. The multiple components to becooled include multiple processor modules to which liquid-cooled coldplates 820 (of a liquid-based cooling system) are coupled, as well asmultiple arrays of memory modules 830 (e.g., dual in-line memory modules(DIMMs)) and multiple rows of memory support modules 832 (e.g., DIMMcontrol modules) to which air-cooled heat sinks are coupled. In theembodiment illustrated, memory modules 830 and the memory supportmodules 832 are partially arrayed near front 831 of electronics drawer813, and partially arrayed near back 833 of electronics drawer 813.Also, in the embodiment of FIG. 8, memory modules 830 and the memorysupport modules 832 are cooled by air flow 815 across the electronicsdrawer.

The illustrated liquid-based cooling system further includes multiplecoolant-carrying tubes connected to and in fluid communication withliquid-cooled cold plates 820. The coolant-carrying tubes comprise setsof coolant-carrying tubes, with each set including (for example) acoolant supply tube 840, a bridge tube 841 and a coolant return tube842. In this example, each set of tubes provides liquid coolant to aseries-connected pair of cold plates 820 (coupled to a pair of processormodules). Coolant flows into a first cold plate of each pair via thecoolant supply tube 840 and from the first cold plate to a second coldplate of the pair via bridge tube or line 841, which may or may not bethermally conductive. From the second cold plate of the pair, coolant isreturned through the respective coolant return tube 842.

FIG. 9 depicts in greater detail an alternate electronics drawer layoutcomprising eight processor modules, each having a respectiveliquid-cooled cold plate of a liquid-based cooling system coupledthereto. The liquid-based cooling system is shown to further includeassociated coolant-carrying tubes for facilitating passage of liquidcoolant through the liquid-cooled cold plates and a header subassemblyto facilitate distribution of liquid coolant to and return of liquidcoolant from the liquid-cooled cold plates. By way of specific example,the liquid coolant passing through the liquid-based cooling subsystem ischilled water.

As noted, various liquid coolants significantly outperform air in thetask of removing heat from heat generating electronic components of anelectronics system, and thereby more effectively maintain the componentsat a desirable temperature for enhanced reliability and peakperformance. As liquid-based cooling systems are designed and deployed,it is advantageous to architect systems which maximize reliability andminimize the potential for leaks while meeting all other mechanical,electrical and chemical requirements of a given electronics systemimplementation. These more robust cooling systems have unique problemsin their assembly and implementation. For example, one assembly solutionis to utilize multiple fittings within the electronics system, and useflexible plastic or rubber tubing to connect headers, cold plates, pumpsand other components. However, such a solution may not meet a givencustomer's specifications and need for reliability.

Thus, presented herein is a robust and reliable liquid-based coolingsystem specially preconfigured and prefabricated as a monolithicstructure for positioning within a particular electronics drawer.

FIG. 9 depicts is an isometric view of one embodiment of an electronicsdrawer and monolithic cooling system, in accordance with an aspect ofthe present invention. The depicted planar server assembly includes amulti-layer printed circuit board to which memory DIMM sockets andvarious electronic components to be cooled are attached both physicallyand electrically. In the cooling system depicted, a supply header isprovided to distribute liquid coolant from a single inlet to multipleparallel coolant flow paths and a return header collects exhaustedcoolant from the multiple parallel coolant flow paths into a singleoutlet. Each parallel coolant flow path includes one or more cold platesin series flow arrangement to cool one or more electronic components towhich the cold plates are mechanically and thermally coupled. The numberof parallel paths and the number of series-connected liquid-cooled coldplates depends, for example on the desired device temperature, availablecoolant temperature and coolant flow rate, and the total heat load beingdissipated from each electronic component.

More particularly, FIG. 9 depicts a partially assembled electronicssystem 913 and an assembled liquid-based cooling system 915 coupled toprimary heat generating components (e.g., including processor dies) tobe cooled. In this embodiment, the electronics system is configured for(or as) an electronics drawer of an electronics rack, and includes, byway of example, a support substrate or planar 905, a plurality of memorymodule sockets 910 (with the memory modules (e.g., dual in-line memorymodules) not shown), multiple rows of memory support modules 932 (eachhaving coupled thereto an air-cooled heat sink 934), and multipleprocessor modules (not shown) disposed below the liquid-cooled coldplates 920 of the liquid-based cooling system 915.

In addition to liquid-cooled cold plates 920, liquid-based coolingsystem 915 includes multiple coolant-carrying tubes, including coolantsupply tubes 940 and coolant return tubes 942 in fluid communicationwith respective liquid-cooled cold plates 920. The coolant-carryingtubes 940, 942 are also connected to a header (or manifold) subassembly950 which facilitates distribution of liquid coolant to the coolantsupply tubes and return of liquid coolant from the coolant return tubes942. In this embodiment, the air-cooled heat sinks 934 coupled to memorysupport modules 932 closer to front 931 of electronics drawer 913 areshorter in height than the air-cooled heat sinks 934′ coupled to memorysupport modules 932 near back 933 of electronics drawer 913. This sizedifference is to accommodate the coolant-carrying tubes 940, 942 since,in this embodiment, the header subassembly 950 is at the front 931 ofthe electronics drawer and the multiple liquid-cooled cold plates 920are in the middle of the drawer.

Referring more particularly to FIGS. 9 & 10A, liquid-based coolingsystem 915 comprises a preconfigured monolithic structure which includesmultiple (pre-assembled) liquid-cooled cold plates 920 configured anddisposed in spaced relation to engage respective heat generatingelectronic components. Each liquid-cooled cold plate 920 includes, inthis embodiment, a liquid coolant inlet 1002 (see FIG. 10A) and a liquidcoolant outlet 1004, as well as an attachment subassembly 1020 (i.e., acold plate/load arm assembly). In a similar manner to the heat sinkattachment approach of FIGS. 3-7, each attachment subassembly 1020 isemployed to couple its respective liquid-cooled cold plate 920 to theassociated electronic component to form the cold plate and electroniccomponent assemblies depicted in FIG. 9. Alignment openings (i.e.,thru-holes) 1010 are provided on the sides of the cold plate to receivealignment pins 332 (FIG. 3) or positioning dowels 1120 (FIG. 11) duringthe assembly process, as described further in the above-incorporatedpatent application entitled “Method of Assembling a Cooling System for aMulti-Component Electronics System”. Additionally, connectors (or guidepins) 1022 are included within attachment subassembly 1020 whichfacilitate use of the attachment assembly, as explained below withreference to FIGS. 11 & 12. Note that load arms 1024 of connectorassembly 1020 are also shown in FIG. 10A.

As shown in FIGS. 9 & 10B, header subassembly 950 includes two liquidmanifolds, i.e., a coolant supply header 952 and a coolant return header954, which in one embodiment, are coupled together via supportingbrackets 1030. In the monolithic cooling structure of FIG. 9, thecoolant supply header 952 is metallurgically bonded in fluidcommunication to each coolant supply tube 940, while the coolant returnheader 954 is metallurgically bonded in fluid communication to eachcoolant return tube 952. A single coolant inlet 951 and a single coolantoutlet 953 extend from the header subassembly for coupling to theelectronics rack's coolant supply and return manifolds (not shown).

FIGS. 9 & 10C depict one embodiment of the preconfigured,coolant-carrying tubes. In addition to coolant supply tubes 940 andcoolant return tubes 942, bridge tubes or lines 941 are provided forcoupling, for example, a liquid coolant outlet of one liquid-cooled coldplate to the liquid coolant inlet of another liquid-cooled cold plate toconnect in series fluid flow the cold plates, with the pair of coldplates receiving and returning liquid coolant via a respective set ofcoolant supply and return tubes. In one embodiment, the coolant supplytubes 940, bridge tubes 941 and coolant return tubes 942 are eachpreconfigured, semi-rigid tubes formed of a thermally conductivematerial, such as copper or aluminum, and the tubes are respectivelybrazed, soldered or welded in a fluid-tight manner to the headersubassembly and/or the liquid-cooled cold plates. The tubes arepreconfigured for a particular electronics system to facilitateinstallation of the monolithic structure in engaging relation with theelectronics system.

To summarize, a cooling system such as disclosed in connection withFIGS. 9-10C advantageously comprises a monolithic structurepreconfigured for actively cooling multiple heat generating electroniccomponents of an electronics system. The monolithic structure includesmultiple liquid-cooled cold plates disposed in spaced relation, witheach liquid-cooled cold plate of the multiple liquid-cooled cold platesbeing configured and positioned to couple to a respective heatgenerating electronic component of the multiple heat generatingelectronic components to be cooled. A plurality of coolant-carryingtubes are metallurgically bonded in fluid communication with multiplecold plates and with a liquid-coolant header subassembly. Theliquid-coolant header subassembly includes a coolant supply headermetallurgically bonded in fluid communication with the multiple coolantsupply tubes and a coolant return header metallurgically bonded in fluidcommunication with multiple coolant return tubes. When in use, themultiple liquid-cooled cold plates are coupled to respective heatgenerating electronic components and liquid coolant is distributedthrough the header subassembly and coolant-carrying tubes to the coldplates for removal of heat generated by the electronic components.

Advantageously, the configuration depicted routes coolant in such amanner as to provide multiple parallel paths through multipleseries-connected liquid-cooled cold plates. This configurationfacilitates maintaining a desired drawer level pressure drop and adesired electronic component level temperature rise. The monolithicstructure is mounted to, for example, the planar circuit board orstiffener via brackets mounted to the header subassembly and a coldplate to electronic component attachment subassembly (see FIGS. 11 & 12)similar to the mounting mechanism depicted and described in detail abovein connection with FIGS. 3-7. The cooling system embodiment depicted isdesigned for direct attachment of the liquid-cooled cold plates to theelectronics component to be cooled, which may include one or more baredies, thereby eliminating the traditional lid and second thermalinterface material.

FIGS. 11 & 12 depict one embodiment of a liquid-cooled cold platedirectly attached to an electronic component comprising multiple baredies residing on a common carrier. As best shown in FIG. 12, the coldplate includes a cold plate base 1200, an active heat transfer region orstructure 1220 and a cold plate lid 1210 having, for example, a coolantinlet 1002 and coolant outlet 1004. The heat generating electroniccomponent 1230 includes, in this example, a carrier 1236 supporting twoprimary heat generating dies 1232 and two secondary heat generating dies1234, each of which is assumed to be a bare die. Additionally, dies 1232are assumed to generate greater heat than dies 1234. In the illustratedembodiment, the active heat transfer structure 1220 of the cold plate isconfigured to reside only over the primary heat generating dies 1232 formore active cooling of the dies compared with dies 1234.

Electronic component 1230 is disposed within a central opening in aloading frame 1100. When in use, loading frame 1100 is affixed to theelectronic system's printed circuit board or planar, and sets theposition for the loading and cooling hardware. Carrier 1236 ofelectronic component 1230 is assumed to be mechanically and electricallycoupled to the printed circuit board as well. A thermal interfacematerial, such as a thermally conductive gel, is disposed between thebare die back sides and the cold plate's contacting surface, whichcontacts the bare dies. Again, the active heat transfer structure 1220of the cold plate is aligned (in this example) only over the highpowered bare dies 1232 (e.g., processor dies). This embodiment seeks tocool the higher power chips preferentially in order to maintain adesired junction temperature in all of the devices being cooled.

The attachment subassembly again includes a pair of load springs 1110connected to load frame 1100. Load frame 1100 is preferably made of analloy material chosen for its low creep properties, such as Zamak 8,while load springs 1110 are preferably made of an alloy material chosenfor its high tensile strength properties, such as a high strength musicwire. Although two load springs 1110 are shown in FIGS. 11 & 12, thoseskilled in the art will appreciate that the present invention may bepracticed with any number of load springs 1110. Load frame 1100 is againmounted to the printed circuit board via fasteners, such as the screwsdescribed above in connection with the embodiment of FIG. 5. Positioningdowels 1120 on either side of the frame engage respective thru-holes1301 (FIG. 13) on either side of the cold plate base 1200. One or morenon-influencing fasteners 1130 are used to secure the cold plate/loadarm assembly to the load frame assembly. By way of example, fournon-influencing fasteners 1130 are mounted on load frame 1100. Thenon-influencing fasteners 1130, which in one embodiment are threadedinto respective bosses of load frame 1100, lock the cold plate inposition without influencing the position of the cold plate in a mannersimilar to that described above in connection with FIGS. 3-6A.

The attachment subassembly again includes load arms 1024 hingedlyconnected via pins 1225 to a U-channel load bracket 1020, which hasopenings to accommodate load transfer block fasteners 1022. Fasteners1022 are threaded at their distal ends to engage respective threadedopenings 1226 in an upper surface of the cold plate base. Load transferblock fasteners 1022 further function as load bracket retaining dowelsin this embodiment. A load transfer block 1221 is disposed below theload bracket 1020 and a load actuation screw 1105 applies compressiveforce to load transfer block 1221, which in turn applies a compressiveload to the cold plate, and hence to the back side of the bare die ofthe electronic component to ensure a desired thermal interface materialthickness, and thus a favorable thermal interface resistance between thebare dies and the contacting surface of the cold plate. As is known, thethermal resistance of the thermal interface material is inverselyproportional to the material's thickness. Advantageously, the cold platebase and load transfer block are configured to distribute loadingpressure across the raised, planar upper surface of the cold plate base.

FIGS. 13 & 13A depict one detailed embodiment of cold plate base 1200.As shown, base 1200 is again configured with active heat transferstructure 1220 extending only over a portion thereof. Within the activeheat transfer structure 1220, multiple parallel channels 1300 aredisposed for passing liquid coolant therethrough. Dowel receivingthru-holes 1301 are provided on either side of the active heat transferstructure for engaging positioning dowels 1120 (FIG. 12). Further,threaded openings 1226 are provided in the upper surface of the coldplate base 1200 and are located to receive respective load transferblock fasteners 1022 (FIG. 12), as described above. A brazing pocket1310 is also shown in FIG. 13A for facilitating brazing of cold platelid 1210 (FIG. 12) to cold plate base 1200. Base cutout areas 1320,which are provided for mass reduction, result in the raised, planarupper surface configuration (when the cold plate lid is attached)illustrated in FIG. 13.

Although preferred embodiments have been depicted and described indetail herein, it will be apparent to those skilled in the relevant artthat various modifications, additions, substitutions and the like can bemade without departing from the spirit of the invention, and that theseare therefore considered to be within the scope of the invention asdefined in the following claims. For example, other non-influencingfastener arrangements may be used in lieu of the non-influencingfastener arrangements described above. Moreover, althoughnon-influencing fasteners may be preferable, adhesives may be used inlieu of the non-influencing fasteners described, such as a pressuresensitive adhesive, UV-sensitive adhesive, thermal curing adhesive,epoxy or any other suitable adhesive.

1. A liquid-based cooling system for cooling an electronics system, thecooling system comprising: a single piece, monolithic structurepreconfigured for cooling multiple heat generating electronic componentsof the electronics system when coupled thereto, the single piece,monolithic structure comprising: multiple liquid-cooled cold platesconfigured and disposed in spaced relation to engage respective heatgenerating electronic components of the multiple heat generatingelectronic components to be cooled; a plurality of coolant-carryingtubes metallurgically, rigidly, permanently bonded in fluidcommunication with the multiple liquid-cooled cold plates; and aliquid-coolant header subassembly metallurgically, rigidly, permanentlybonded in fluid communication with multiple coolant-carrying tubes ofthe plurality of coolant-carrying tubes, the liquid-coolant headersubassembly comprising a coolant supply header bonded in fluidcommunication with multiple coolant supply tubes of the multiplecoolant-carrying tubes and a coolant return header bonded in fluidcommunication with coolant return tubes of the multiple coolant-carryingtubes, wherein when in use, the multiple liquid-cooled cold platesengage the respective heat generating electronic components of themultiple heat generating electronic components, and liquid coolant isdistributed through the liquid-coolant header subassembly and pluralityof coolant-carrying tubes to the multiple liquid-cooled cold plates forremoval of heat generated by the respective heat generating electroniccomponents.
 2. The cooling system of claim 1, wherein the monolithicstructure is preconfigured with the cold plates being in multiple setsof series-connected liquid-cooled cold plates, each set ofseries-connected liquid-cooled cold plates comprising n series-connectedliquid-cooled cold plates, wherein n≧2, and wherein multiple parallelcoolant flow paths are defined within the monolithic structure throughthe multiple sets of series-connected liquid-cooled cold plates when thecooling system is operational.
 3. The cooling system of claim 2, whereinthe n series-connected liquid-cooled cold plates of each set of nseries-connected liquid-cooled cold plates are coupled in series fluidcommunication employing a respective set of coolant-carrying tubes ofthe plurality of coolant-carrying tubes, each respective set ofcoolant-carrying tubes comprising: a coolant supply tube metallurgicallybonded between the coolant supply header of the liquid-coolant headersubassembly and a first liquid-cooled cold plate of the respective setof n series-connected liquid-cooled cold plates; at least one bridgetube connecting a liquid-cooled cold plate of the n-series-connectedliquid-cooled cold plates to another liquid-cooled cold plate of therespective set of n series-connected liquid-cooled cold plates; and areturn tube metallurgically bonded between an n^(th) liquid-cooled coldplate of the respective set of n series-connected liquid-cooled coldplates and the coolant return header of the liquid-coolant headersubassembly.
 4. The cooling system of claim 3, wherein the bridge tubeof each respective set of coolant-carrying tubes is metallurgicallybonded between the first liquid-cooled cold plate and the anotherliquid-cooled cold plate, and wherein the metallurgical bonding of eachrespective set of coolant-carrying tubes to the liquid-coolant headersubassembly and to the respective set of n series-connectedliquid-cooled cold plates is one of a weld, braze or solder bonding. 5.The cooling system of claim 1, wherein the multiple coolant-carryingtubes metallurgically bonded to the liquid-coolant header subassemblycomprise multiple thermally conductive coolant-carrying tubespreconfigured to facilitate coupling of the monolithic structure to themultiple heat generating electronic components of the electronicssystem, and wherein the multiple thermally conductive coolant-carryingtubes are each metallurgically bonded in fluid communication to theliquid-coolant header subassembly employing one of a fluid-tight weld,braze or solder bond.
 6. The cooling system of claim 5, wherein eachthermally conductive coolant-carrying tube is metallurgically bonded influid communication to at least one respective liquid-cooled cold plateof the multiple liquid-cooled cold plates employing one of a fluid tightweld, braze or solder bond.
 7. The cooling system of claim 1, whereineach heat generating electronic component to be cooled comprises atleast one bare die, and wherein a surface of each liquid-cooled coldplate of the multiple liquid-cooled cold plates is configured todirectly attach to the at least one bare die of the respective heatgenerating electronic component.
 8. A cooled electronic systemcomprising: an electronics drawer of an electronics rack, theelectronics drawer containing multiple heat generating electroniccomponents to be cooled; and a liquid-based cooling system for coolingthe multiple heat generating electronic components of the electronicsdrawer, the cooling system comprising: a single piece, monolithicstructure preconfigured for the electronics drawer and coupled to themultiple heat generating electronic components of the electronicsdrawer, the single piece, monolithic structure comprising: multipleliquid-cooled cold plates preconfigured in spaced relation and coupledto respective heat generating electronic components of the multiple heatgenerating electronic components to be cooled; a plurality ofcoolant-carrying tubes metallurgically, rigidly, permanently bonded influid communication with the multiple liquid-cooled cold plates; and aliquid-coolant header subassembly metallurgically, rigidly, permanentlybonded in fluid communication with multiple coolant-carrying tubes ofthe plurality of coolant-carrying tubes, the liquid-coolant headersubassembly comprising a coolant supply header bonded in fluidcommunication with multiple coolant supply tubes of the multiplecoolant-carrying tubes and a coolant return header bonded in fluidcommunication with coolant return tubes of the multiple coolant-carryingtubes, wherein in operation, liquid coolant is distributed through theliquid-coolant header subassembly and plurality of coolant-carryingtubes to the multiple liquid-cooled cold plates for removal of heatgenerated by the respective heat generating electronic components of theelectronics drawer.
 9. The cooled electronic system of claim 8, whereinthe multiple coolant-carrying tubes metallurgically bonded to theliquid-coolant header subassembly comprise multiple thermally conductivecoolant-carrying tubes preconfigured to facilitate coupling of themonolithic structure to the multiple heat generating electroniccomponents of the electronics drawer, and wherein the multiple thermallyconductive coolant-carrying tubes are each metallurgically bonded influid communication to the liquid-coolant header subassembly employingone of a fluid-tight weld, braze or solder bond.
 10. The cooledelectronic system of claim 9, wherein each thermally conductivecoolant-carrying tube is metallurgically bonded in fluid communicationto at least one respective liquid-cooled cold plate of the multipleliquid-cooled cold plates employing one of a fluid tight weld, braze orsolder bond.
 11. The cooled electronic system of claim 8, wherein eachheat generating electronic component to be cooled comprises at least onebare die, and wherein a surface of each liquid-cooled cold plate of themultiple liquid-cooled cold plates is directly attached to the at leastone bare die of the respective heat generating electronic component. 12.The cooled electronic system of claim 11, wherein at least one heatgenerating electronic component to be cooled comprises multiple baredies, and wherein the surface of the liquid-cooled cold plate coupledthereto is directly attached to the multiple bare dies, the multiplebare dies comprising at least one primary heat generating die and atleast one secondary heat generating die, and wherein the liquid-cooledcold plate coupled thereto comprises liquid-cooled channels extendingover the at least one primary heat generating die without extending overthe at least one secondary heat generating die.
 13. The cooledelectronic system of claim 12, wherein the at least one primary heatgenerating die comprises at least one processor die and the at least onesecondary heat generating die comprises at least one non-processor die,and wherein each heat generating electronic component to be cooledcomprises multiple bare dies, the multiple bare dies of each heatgenerating electronic component comprising the at least one primary heatgenerating die and the at least one secondary heat generating die, andwherein liquid-cooled channels of each liquid-cooled cold plate of themonolithic structure extend over the respective at least one primaryheat generating die without extending over the respective at least onesecondary heat generating die.
 14. The cooled electronic system of claim8, wherein the monolithic structure further comprises multipleattachment subassemblies, each attachment subassembly being configuredto mount a respective liquid-cooled cold plate to its respective heatgenerating electronic component, and wherein each liquid-cooled coldplate comprises a raised, planar upper surface and each attachmentsubassembly comprises a load transfer block disposed between a loadingbracket and the raised, planar upper surface of the respectiveliquid-cooled cold plate, the load transfer block being configured todistribute loading pressure across the raised, planar upper surface ofthe respective liquid-cooled cold plate.
 15. A cooled electronics systemcomprising: an electronics rack comprising at least one electronicsdrawer, the at least one electronics drawer having a component layoutcontaining multiple heat generating electronic components to be cooled;and a liquid-based cooling system for cooling the multiple heatgenerating electronic components of the electronics drawer, theliquid-based cooling system comprising: a single piece, monolithicstructure preconfigured for the component layout of the at least oneelectronics drawer and coupled to the multiple heat generatingelectronic components thereof, the single piece, monolithic structurecomprising: multiple liquid-cooled cold plates preconfigured in spacedrelation and coupled to respective heat generating electronic componentsof the multiple heat generating electronic components to be cooled; aplurality of coolant-carrying tubes metallurgically, rigidly,permanently bonded in fluid communication with the multipleliquid-cooled cold plates; and a liquid-coolant header subassemblymetallurgically, rigidly, permanently bonded in fluid communication withmultiple coolant-carrying tubes of the plurality of coolant-carryingtubes, the liquid-coolant header subassembly comprising a coolant supplyheader bonded in fluid communication with multiple coolant supply tubesof the multiple coolant-carrying tubes and a coolant return headerbonded in fluid communication with coolant return tubes of the multiplecoolant-carrying tubes, wherein in operation, liquid coolant isdistributed through the liquid-coolant header subassembly and pluralityof coolant-carrying tubes to the multiple liquid-cooled cold plates forremoval of heat generated by the respective heat generating electroniccomponents of the electronics drawer.